Click any tag below to further narrow down your results
Links
David Breunig's talk on how context engineering emerged after the long-context push of 2024, why bigger context windows introduced new failure modes, and the six patterns builders now use to manage what goes into a model's context. It's a useful map of where the practice has been and where it's headed.
- Million-token context windows didn't solve the problem. Long contexts add room for errors as well as information, leading to failure modes like context poisoning, distraction, confusion, and clash.
- Gemini 2.5 playing Pokémon shows poisoning in practice: a fact baked into its weights (the tea bribe from the older game) got stuck in its context and cost it hours. Performance also degraded past roughly 100,000 tokens, with the agent repeating old actions instead of planning new ones.
- Breunig's research cites shows instruction-following drops as bad examples pile up in context, affecting every model tested, including smaller ones that fall below half adherence after a few contradictions. The six-tool toolbox (augmentation, loadout, quarantine, pruning, offloading, summarization) has stayed stable as these problems emerged.
@0xMortyx on X
A data scientist ran a blind test pitting RAG against Kimi K3's full 1M-token context window on the same 12 questions. Long context scored perfectly on answer quality but cost about 16 times as much and ran roughly 3 times slower, so the post argues the right choice depends on corpus size and query volume.
- Long context answered all 12 questions perfectly, while RAG was near-perfect but weaker on completeness.
- Long context cost about 16 times as much and took roughly 3 times as long per question; at 12,000 queries the cost gap reaches about $3,800 versus $230.
- Skipping RAG is only backed by evidence when the corpus sits well under a fifth of the window; the test never covered corpora filling 60-80% of it.
- For tasks spanning many unconnected sources, the author points to Kimi K3's Agent Swarm and a connected context graph instead of either approach.